# 4096 token length

A medical knowledge-enhanced model based on the BigBird architecture, further pretrained with MIMIC-III clinical records, supporting input lengths of up to 4096 tokens, outperforming ClinicalBERT on multiple medical NLP tasks.

A long-sequence processing model based on XLM-R extension, supporting sequences up to 4096 tokens, suitable for multilingual tasks

A clinical knowledge-enhanced model based on the Longformer architecture, supporting input lengths of up to 4096 tokens, outperforming ClinicalBERT by at least 2 percentage points on 10 benchmark datasets.

Extended XLM-RoBERTa model supporting sequences up to 4096 tokens, suitable for multilingual tasks

ParsBigBird is a Persian Bert model based on the BigBird framework, supporting text sequences up to 4096 tokens in length